Method of determining the position of an object in an image

ABSTRACT

The invention relates to a method of determining the position of an object in an (X-ray) image (1). A pattern of marking elements is attached to the object., wherein the marking elements are not visibly evident individually in the image (I), i.e. they form an invisible “watermark”. By means of a correlation between the image (I) and a filter image (M) of the pattern of marking elements, however, the position of the marking elements, and thereby that of the object, can be localized in the image (I).

The invention relates to a method of determining the position of anobject in an image, to marking means for application in a method of thiskind, and to an X-ray system set up to implement the method.

Images of objects are generated in many different application areas andsubsequently analyzed in respect of particular investigations. Inanalyses of this kind, it is frequently important to be able to localizethe position of at least one object or of a particular site of an objecton the image. The taking of medical X-ray images is considered below asan example to illustrate investigations of this kind. In order toexecute a precise navigation of a catheter, for example, with the aid ofX-ray images, or to relate various images from a sequence to one anotherwith a high degree of accuracy (typically in the sub-mm range), it mustbe possible to localize the examination table and/or organs of thepatient as precisely as possible on the X-ray images. It is true that,in principle, the position of the examination table relative to theX-ray apparatus can be detected by position sensors, such aspotentiometers. However, the inaccuracy of these is generally much toogreat to enable the necessary pixel-precise localization of the table.

For a more precise determination of the positions of objects in X-rayimages, therefore, a method is used in which X-ray-absorbent markingelements of known size, shape and absorption properties, which can beclearly recognized on the X-ray image, are attached to the examinationtable and/or the patient. However, the disadvantage of this is that themarking means distort the actual representation of the body. For thisreason, they are generally used outside the observation field ofinterest, which leads in turn to a decrease in precision in the mostimportant region of the image and to an enlargement of the area exposedto radiation.

In order to cover the image gaps caused by marking elements, it is knownfrom WO 00/00086 for the sites of the marking means to be filled by aninterpolation of the surrounding image content in a post-processing ofthe X-ray image. However, this can only succeed for marking elementsthat are small in size, and also carries the risk that image informationthat is not in fact present may be simulated at the sites of the markingmeans. In the case of semi-transparent marking elements, it is furtherknown from WO 00/00086 for the absorption through the marking elementsto be calculated subsequently in order to show the translucent imagecontent in the area of the marking elements, undiminished and to besteffect. Here again, however, a certain loss of image information atlocations of weak exposure is unavoidable.

Against this background, it is an object of the present invention toprovide means for determining the position of an object in an image,which means enable a precise localization of the object withoutimpairment of the image.

This object is achieved by a method with the features as claimed inclaim 1, by marking means with the features as claimed in claim 6 and byan X-ray system with the features as claimed in claim 9. Advantageousembodiments are contained in the dependent claims.

The method in accordance with the invention serves for determining theposition of an object in an image. An image may be, in particular, anX-ray image, wherein the method is, however, not confined to this area,and also covers, for example, images generated by ultrasound, magneticresonance, scintigraphy, photography or similar. It is characteristic ofthe method that a (coherent or fragmented) pattern of marking elementsis attached to the object to be localized, wherein the marking elementsare not visibly evident individually in the image. In this context, “notvisibly” means that, during evaluation of the image, the markingelements, each considered per se, are not detectable or distinguishablefrom the remaining image content. In particular, they cannot berecognized when the image is visually evaluated by a human observer.This may be achieved by, for instance, making the marking elements smallenough that they affect only one or just a few pixels. Additionally oralternatively, the difference brought about by a marking means at alocation of the image from an image without marking means may be sosmall that it remains in the range of the background image noise. Oneindividual image element cannot therefore be distinguished from thebackground noise of the image. In an X-ray image, for example, this maybe achieved by marking means with very low absorption coefficients,which scarcely change the developing X-ray image.

With the proposed method, a “watermark”, which is invisible in standardimage analysis and which does not distort or impair the normal image, isthereby introduced. The “watermark” may therefore be sited at any randomlocation of the image, so that, in particular, an object in the part ofthe image of interest in the center can be marked and localized.

Whereas the individual marking element is not visible in the image and(without knowledge of the pattern) cannot be reliably localized, thepattern comprising multiple marking elements, which is assumed to beknown, enables the common localization of all, or multiple, markingelements with the aid of suitable analysis methods. In accordance with apreferred embodiment, the position of the marking elements in the imageis hereby determined by a correlation of the image with at least oneso-called “filter image”, which represents the (fundamental) pattern ofthe marking elements. The filter image of the pattern has a maximumcorrelation with the image to be investigated when it lies preciselyabove the pattern of the marking elements concealed in the image. Inother positions, on the other hand, the filter image will exhibit only avery much lower, randomly-related correlation. From the particularrelative position between the image and the filter image that leads to amaximum correlation, the position of the marking elements and of theobject to be localized can be determined in the image by means of thecorrelation method.

A bandpass filtration in the frequency range of the pattern ispreferably undertaken before the correlation method in order to remove,as thoroughly as possible, the image components that are independent ofthe pattern.

In the correlation method described above, the filter image of thefundamental pattern may be transformed relative to the actual pattern(“original pattern”) of the marking elements, i.e. does not have toreflect this identically. In preferred transformations, the pattern isscaled, i.e. proportionately enlarged or reduced, rotated and/ordistorted. Through the use of filter images which have been transformedrelative to the original pattern of the marking elements, account may betaken of corresponding transformation of the original pattern whengenerating the image. For instance, with a conventional X-ray image, anoriginal pattern of marking elements attached to the examination tableparallel with the image sensor plane of the detector will be shown(slightly) enlarged in the image owing to the projection circumstances.The enlargement factor here depends on the geometry, in particular thedistances between the X-ray source, the marking elements and the imagesensor plane. This geometry will generally be known a priori withsufficient accuracy, so the associated scaling factor can betheoretically calculated and used for generating the filter image. If,however, the geometry is unknown or requires verification, thecorrelation method explained above may be undertaken with filter imagesof different scaling factors. The scaling that matches best will herebyshow the largest value of the maximum correlation, so the geometry ofthe imaging configuration can also be determined in this manner. Similarconsiderations also apply to determination of general transformations,such as distortions of the original pattern that have arisen as a resultof an oblique or freely curved position of the original pattern relativeto the image sensor plane. Even these distortions can be identified bythe filter image with the largest value of the maximum correlation.

The filter image of the pattern may be generated, by calculation, fromthe knowledge of the original pattern. It is, however, also conceivablefor the filter image to be generated empirically by a separate pictureof the marking means, wherein as few other objects as possible, or noneat all, should be present in the area of the image and the imagingparameters must be selected in such a way that the marking elements can(exceptionally) be localized individually in this particular image.

In a preferred application area for the method, the image is generatedby means of radioscopy, wherein the marking elements exhibit anabsorption of the X-rays so low that the effect of this absorption lieswithin the noise level of the X-ray image. The individual markingelement can therefore not be detected, either visually or with automaticanalysis methods without knowledge of the original pattern. The normalimage is therefore not impaired by the marking elements.

In accordance with a development of the method, the position of at leastone further object is determined in the image in that a second patternof marking elements, which, like the marking elements of the firstpattern, do not show up individually in the image, is attached to thefurther object. The second pattern is, furthermore, different from thefirst pattern, so the first and second patterns, and thereby the objectsmarked hereby, can be distinguished from one another. In principle, aquasi-arbitrary number of different objects may be localized in thismanner without visible distortions of the normal image occurring. Atypical application example is that of the parallel localization of anexamination table and of a patient lying on it in an X-ray image.

The invention further relates to a marking means provided for attachingto an object in order to determine its position in an image of theobject (and, if applicable, of further objects). The marking meanscomprise marking elements arranged in a pattern, wherein the markingelements themselves are designed in such a way that they are not visiblyevident individually in the image. The marking means are therebysuitable to be used in a method of the kind explained above. Theadvantages and variation options of the method are thereforeappropriately assured for the marking means.

The marking elements are preferably applied to a transparent carrier,such as a foil, so that the pattern formed by them is fixed and themarking means can be handled as easily as possible. The “transparency”of the carrier hereby means, in a generalized sense, that it does notappear on the image to be generated, i.e. in the case of an X-ray image,for example, it exhibits an imperceptibly low absorption forX-radiation.

The pattern formed by the marking elements may be completely randomprovided that, on the basis of its pattern characteristic, it enablesthe subsequent, coordinated localization of the marking elements in animage. The pattern may, in principle, also be formed by a coherent areaof touching or overlapping marking elements. However, the markingelements are preferably arranged in isolation from one another anddistributed over a large surface in order to prevent distorting effectson the image and to ensure the greatest possible precision of thelocalization. It is especially preferred if the pattern of the markingelements shows a good correlation behavior in the sense that thecorrelation of the pattern with itself is high only when there isprecise superimposition, and otherwise is low in all offset positions. Agood correlation behavior of this kind is exhibited by, in particular, atwo-dimensional maximum-length sequence. A maximum-length sequence is abinary sequence (i.e. only values 0 and 1 are possible) with a period2^(r−)1 with rεN.

The invention further relates to an X-ray system comprising thefollowing elements:

An X-ray source, which can preferably emit X-radiation conically.

An X-ray detector, which is disposed in the ray path of the X-ray sourceand is equipped with image sensors to measure the incident radiationdose.

At least one marking means for attachment to an object located betweenthe X-ray source and the X-ray detector in order to determine itsposition in an X-ray image, wherein the marking means comprise markingelements arranged in a pattern, which are not visibly evidentindividually in the X-ray image.

A data processing unit for calculation of the position of the markingmeans in an image generated with the X-ray system.

For such a case of an X-ray image, the above-mentioned method can beimplemented with an X-ray system of this kind. Therefore, the advantagesof this method can also be achieved, i.e. an extremely accuratepositional determination of objects in an X-ray image without anyvisible impairment of the X-ray image by marking elements.

The X-ray system is preferably of a design such that it can implementone or more variants of the method. For example, the data processingunit may be set up to calculate a correlation between the X-ray imageand at least one filter image of the pattern of the marking elements,wherein the filter image may have been transformed relative to theoriginal pattern of the marking elements. The marking elements may,furthermore, exhibit an absorption for X-radiation so low that theireffect lies within the noise level of the X-ray image. The X-ray systemmay also comprise at least two marking means, which can be attached todifferent objects in the ray path of the X-ray source and which comprisedifferent patterns of marking elements.

The invention will be further described with reference to examples ofembodiments shown in the drawings, to which, however, the invention isnot restricted.

FIG. 1 shows an X-ray system set up for implementing the invention.

FIG. 2 shows the principle of positional determination of an object inan X-ray image generated with the X-ray system from FIG. 1.

The essential components of an X-ray system shown schematically in FIG.1 comprise an X-ray source 1, opposite which an X-ray detector 3 with adetector surface comprising image sensors (not shown) is disposed . TheX-ray source 1 and the X-ray detector 3 are generally disposed in afixed relative-geometry on e.g. a C-arm (not shown). Also identifiableis a data processing unit 2, which is coupled with the X-ray source 1and the X-ray detector 3 in order to drive these and in order to receiveand further process X-ray images I taken by the X-ray detector 3. Thedata processing unit 2 is generally coupled with output devices, such asa monitor, in order to represent the X-ray image to a user. The X-raysystem further comprises an examination table 4, on which a patient tobe X-rayed (not shown) can lie.

If, for example, a catheter investigation is to be observedfluoroscopically with the X-ray system, X-ray images produced at varioustimes have to be related to one another as accurately as possible. Tothis end, it must be possible to localize marked objects, such as theexamination table 4 or points of the patient's body, on the variousX-ray images with a high degree of accuracy.

Similar requirements also apply in the case of computer-tomographicX-ray images, in which the X-ray source 1 and the X-ray detector 3rotate around the patient helically. In order to execute the intendedthree-dimensional reconstruction of the imaged body volume, it must bepossible to relate the X-ray images taken from various directions to oneanother with pixel accuracy. The measurement accuracy of the sensors onthe carriers of the X-ray source 1 and the X-ray detector 3, or on theinvestigation table 4, is, in the majority of cases, not adequate. Forthis reason, it is important to be able to identify marked points on theimages themselves with a high degree of precision.

In order to fulfill the above-described requirements, the use of aspecial marking means 5 is provided in accordance with the invention. Inthe example in FIG. 1, the marking means 5 comprises a (virtually)X-ray-transparent (metallic) foil, which carries a pattern of“dot-shaped” marking elements 6, which are shown, greatly enlarged, inFIG. 1. The marking elements 6 preferably comprise a material extremelyimpervious to X-rays, such as copper or gold, with a small layerthickness. The marking elements 6 are therefore, regarded absolutely,only very slightly absorbent and mechanically flexible. They are also ofa size that is as small as possible, which is preferably selected suchthat a marking element 6 approximately covers the area of an imagesensor in the X-ray detector 3. The marking elements 6 thereforetypically have a diameter in the range from approximately 100 μm to 1000μm, wherein especially preferred is a diameter of approximately 150 μm.

Different methods come into consideration for producing the markingmeans 5. For instance, a Cu layer may firstly be applied to X-ray thinmaterial, such as a polymethacrylate (“Plexiglas™”). The desired patternof the marking elements can subsequently be formed in the Cu layer usinglithographic methods, by thermal ablation (lasers etc.), or similar.Desired patterns may also be produced by stamping or boring out acarrier, by the vapor-deposition of substances onto a carrier or by amultiplicity of other methods.

By virtue of the shape, size, thickness and material selection of themarking element 6, the attenuation of the X-radiation is so small thatit is concealed even under the most unfavorable conditions of systemnoise. The marking elements 6 are thereby not visibly evident on anX-ray image I.

As described below with reference to FIG. 2, the marking elements can,however, be localized in an X-ray image with the aid of suitable methodsbased on the knowledge of the pattern. A method of this kind starts froman X-ray image I, which has been created with the X-ray system of FIG. 1and which contains the hidden pattern of marking elements as a“watermark”. Further stored in the data processing unit 2, whichexecutes the method, is a filter image M, which reflects the fundamentaloriginal pattern of the marking elements 6 (i.e. without other objects).The filter image M may hereby identically replicate the original patternof the marking elements 6, but may also contain it in scaled (enlargedor reduced), rotated or distorted form if corresponding transformationsof the original pattern can be anticipated in the X-ray image I. Thefilter image M may, in particular, represent the original pattern in anenlargement that derives from the underlying imaging geometry of theX-ray system.

The pattern is preferably selected from the family of two-dimensional,cyclical binary maximum-length sequences (see K. D. Lüke,Korrelationssignale, (Correlation Signals), Springer-VerlagBerlin-Heidelberg, 1992, chapter 3.4), wherein the period of thesequence is half as great in each direction as the correspondingdetector size in this direction, and wherein a “1” of the sequenceindicates the presence, and a “0” of the sequence indicates the absenceof a marking element. A (one-dimensional) maximum-length sequence I_(n)(nεN) is a binary sequence with a period 2^(r −)1 with rεN. A“two-dimensional maximum-length sequence” I_(n,m) (n, mεN) is defined inthat, for each fixed n₀εN, the one-dimensional sequence I_(n) _(o,) _(m)(mεN) is a one-dimensional maximum-length sequence and, conversely, foreach fixed m_(o)εN, the one-dimensional sequence I_(n,m) _(o) (nεN) is aone-dimensional maximum-length sequence. Patterns formed frommaximum-length sequences exhibit an especially good correlationbehavior, i.e. the correlation of the pattern with its copy is highgiven an identical position of the copy, whereas, in all mutually offsetpositions of the pattern and copy, it is considerably lower, and, forexample, fluctuates around a low average value.

In addition to the maximum-length sequences, other sequences with goodcorrelation behavior known from the relevant literature are, of course,especially suitable for the pattern formation. The sequences used forpattern formation do not necessarily have to be binary hereby. Forexample, trivalent, quadrivalent, quinquevalent or higher-valuesequences may be simulated by marking elements of different “strengths”(i.e. degree of absorption). Technically, marking means of this kind maybe realized by e.g. coating a carrier with a metal (Cu, Au etc.) withlocally differing frequencies or thicknesses.

In order to localize the position of the original pattern of the markingelements in the X-ray image I, the X-ray image I is correlated, point bypoint, with the filter image M. For images constructed from pixels inmatrix form, this operation can be expressed mathematically as follows:${{P\left( {x,y} \right)} = {\sum\limits_{x^{\prime},y^{\prime}}^{\quad}\quad{{{I\left( {x^{\prime},y^{\prime}} \right)} \cdot {M\left( {{x^{\prime} - x},{y^{\prime} - y}} \right)}}{\forall x}}}},y$wherein I(x, y), M(x, y) and P(x, y) denote the pixel values of theX-ray image I, of the filter image M and of the calculated product imageP in the pixel (x, y).

In every point (x, y) of the X-ray image I, the correlation between thefilter image M and the X-ray image I yields a virtually identical(average) value. One important exception here, however, is the pointC=(x_(c,) y_(c)), at which, during the correlation operation, the filterimage M lies precisely above the X-ray image I in such a way that itcoincides with the concealed pattern of the marking elements 6. Inrespect of the above formula, this means that one point (x′, y′) of theX-ray image I belongs to precisely one marking element 6 when (x′−x_(c),y′−y_(c) belongs to the same marking element in the filter image M. Inthe case where the filter image M is in a position of this kind, amaximum correlation thereby arises between the original pattern and thepattern of the filter image M, which leads to a maximum value of thecorrelation sum P(x_(c), y_(c)). In the image P, this point, whichcorresponds to the center C of the pattern, is thereby clearlydetectable. The position of the marking elements in the X-ray image Ican therefore also be determined via the position of this point C, as aresult of which, in turn, the position of an object in fixed connectionwith the marking elements, such as the investigation table 4, can beidentified. In accordance with FIG. 2, the localized object can then beshown highlighted in a new X-ray image I*.

Furthermore, the known position of the pattern of the marking elements 6in the X-ray image I can also be used to calculate the known (weak)absorption of the marking elements 6 in order to minimize any imagechange resulting from the marking elements 6.

In accordance with a variant of the above method, a high-pass filtrationis executed before and/or after the correlation. As a result, slowlyvarying components of the image, which may stem from the imaging of theactual object and may distort the positional determination of theoriginal pattern, can be removed.

In principle, any (original) pattern whatever of marking elements 6 maybe used for the method. In particular, a pattern may also containinformation such as lettering and/or illustrations (e.g. a companylogo). In the simplest case, this information may be imaged directly(geometrically) in the pattern. Preferably, however, it is implicitlyimplemented in the pattern in such a way that it is not identifiableuntil the post-processing. For instance, a pattern may be composed ofthe multiple superimposition of a shifted two-dimensional maximum-lengthsequence M. During the subsequent correlation of this composed patternwith the two-dimensional maximum-length sequence as the filter image M,a distinct point is always generated when the filter image M liesprecisely above a maximum-length sequence contained in the pattern. Withappropriate construction of the pattern, virtually any desired image canbe composed from the points generated in this manner.

In many applications, it is necessary, additionally or alternatively tothe positional determination of the examination table 4 shown in FIG. 1,to monitor any independent movement of the patient. In this case, amarking means is provided in the form of a foil with similar markingelements to the marking means 5 of the table, and this is secured to thepatient's back. The various marking means preferably hereby exhibitdifferent patterns. In particular, the two-dimensional maximum-lengthsequences of the various marking means may be selected from a family oforthogonal codes, so that the results of a mutual correlation do notinterfere with each other.

1. A method of determining the position of an object in an image,wherein a pattern of marking elements, which are not visibly evidentindividually in the image, is attached to the object.
 2. A method asclaimed in claim 1, wherein the position of the marking elements in theimage is determined by a correlation of the image with at least onefilter image of the pattern of the marking elements.
 3. A method asclaimed in claim 2, wherein the filter image of the pattern istransformed relative to the actual pattern of the marking elements.
 4. Amethod as claimed in claim 1, wherein the image is generated by means ofradioscopy, and the marking elements exhibit a low absorption of theX-rays, the effect of which lies within the noise level of the X-rayimage.
 5. A method as claimed in claim 1, wherein the position of atleast one further object is determined in the image, wherein a secondpattern of marking elements, which do not show up individually in theimage, is attached to the further object, and wherein the second patternis different from the first pattern.
 6. Marking means for attaching toan object in order to determine its position in an image, wherein themarking means comprise marking elements arranged in a pattern, which arenot visibly evident individually in the image.
 7. Marking means asclaimed in claim 6, wherein the marking elements are applied to atransparent carrier.
 8. Marking means as claimed in claim 6, wherein thepattern of marking elements 6 his a two-dimensional maximum-lengthsequence.
 9. An X-ray system, comprising an X-ray source generating aray path; an X-ray detector, which is disposed in the ray path of theX-ray source; at least one marking means for attachment to an object inorder to determine the position of the object in an X-ray image, whereinthe marking means comprise marking elements, which are not visiblyevident individually in the X-ray image. a data processing unit forcalculation of the position of the marking means in an image generatedwith the X-ray system.
 10. An X-ray system as claimed in claim 9,wherein it is set up to implement a method as claimed in claim
 1. 11.The X-ray system as claimed in claim 9, wherein said marking elementsare arranged in a pattern.
 12. The X-ray system as claimed in claim 9,wherein said pattern is a two dimensional, cyclical binary maximumlength sequence.
 13. The X-ray system as claimed in claim 9, whereinsaid marking elements are applied to a transparent carrier.